Constrained Fine-Grain Parallel Sparse Matrix Distribution

We consider how to distribute sparse matrices among processors to reduce communication cost in parallel sparse matrix computations, in particular, sparse matrix-vector multiplication. We allow 2d distributions, where the distribution (partitioning) is not constrained to just rows or columns. The fine-grain model is a 2d distribution introduced in [2] where nonzeros can be assigned to processors in an arbitrary general way. They proposed a hypergraph model and showed it can significantly reduce the communication volume compared to 1d distributions. We define a constrained version of this problem, where the input and output vector distributions are given. We propose two combinatorial models. The first is based on vertex cover in the bipartite graph, and the second on hypergraph partitioning with fixed vertices. Though NP-hard, both models can be solved heuristically using existing algorithms and software. Sparse matrix-vector multiplication is usually parallelized such that the processor that owns element aij computes the contribution aijxj . This is a local operation if xj , yi and aij all reside on the same processor; otherwise communication is required. In general, the following four steps are performed: